BOOTP and Beowulf

BOOTP Overview

BOOTP (IETF RFC 951) is a bootstrapping protocol designed to allow stateless hosts (clients) attached to a TCP/IP network to learn their configuration information from some host (server) endowed with knowledge about the nodes on its network.

BOOTP is only a lightly extensible protocol with severe restrictions on reply message size (if you choose not to use extension files). While BOOTP can, in principle, provide clients with information beyond TCP/IP configuration, the Beowulf project uses it only to distribute TCP/IP configuration.

BOOTP Server-side

We use the CMU bootpd v2.4 (this and newer versions for Linux) It is provided by most of the standard Linux distributions, including RedHat. There is no reason that we couldn't write a master Beowulf-server daemon that provided bootp server functionality based on a master Beowulf database rather than rely on bootptab files and external daemons. However, we don't really feel that this will simplify your life (for two reasons):

There's already a solution that works (bootpd)
You'd rather have your new Beowulf nodes get down to the serious business of clustered computing rather than stooge around serving BOOTP to one another. Add another network interface to the same crusty SPARCServer/Apollo/DECStation that serves BOOTP to the rest of you enterprise and pretend it's a worldly node in your cluster.
Drexel University's Beowulf cluster is configured in precisely this way. Their 17 node cluster is connected via a private Fast Ethernet interface to their departmental SPARCServer/1000.

BOOTP Client-side

BOOTP provides a lot of client-side flexibilty and we've chosen to use it in the way that retains a lot of that flexibility.
See the description of the Beowulf node-boot sequence to see how BOOTP fits in; nodes may actually perform the BOOTP step in several ways:

A node may have some PROM based facility for booting from a network. We expect these facilities to be fairly crude and support only the standard tags that relate to basic network configuration.
A node may have some custom program it runs from a small hard drive. This may be a sophisticated boot loader that performs the BOOTP request, gathers the result, and passes control to either a local operating system image or to one it retrieved from the net.
A node may have an entirely local copy of its operating system (Linux, in our case) and simply run one of the many available bootp client programs as part of an initialization script.
Earlier Beowulf clusters work in this way.

Notes: (Q and A)

Q: My BOOTP packets NEVER get recieved by my BOOTP server

A: BOOTP is a UDP protocol. It is NOT reliable. For instance:
Under Linux, many 10/100Mbps Ethernet cards do not switch to 100MBps until a packet is sent out. If your node is connected to a 100Mbps Ethernet repeater, your Ethernet card didn't have a link until the card switched to 100Mb mode. If the first packet to the network in the boot sequence is an unreliable UDP packet (i.e. a BOOTP packet), that packet might be lost to the world. Make sure you pick a BOOTP client that will retry at least once before giving up.

A.2: Modern experimental kernels silently discard packets from 0.0.0.0 (ala BOOTP requests) unless you twiddle with the /proc/sys/net interface. See your linux/Documentation tree for details.

Q: My worldly node running the BOOTP server is connected to an extremely busy external network. There are 115 Xterminals and 25 network printers barraging me constantly with bootp requests. I only want to make bootp available to the private networks.

A: If your BOOTP server is a linux box with IP firewalling, try the following:

	/sbin/ipfwadm -I -a deny -P udp -D 0.0.0.0/0 bootps -W eth1

for example. The above will deny all bootp packets arriving on the eth1 interface. Supply the name of your external interface in place of eth1 for your situation.

Q: I want to run a 'private' bootpd for my cluster on a worldly node. Can I do this while also running a 'public' bootpd.

A: If you wish to do this (perhaps because you really want to have a 'private' bootptab that can be safely machine-written), you will need a kernel with IP firewalling and transparent proxy support. Try the following:

	/sbin/ipfwadm -I -a accept -P udp -D 0.0.0.0/0 bootps -W eth0 -r 2015

The above will route all bootp requests arriving on the eth0 interface to port 2015 on the same machine. You can configure your 'private' bootp server to listen to this port for requests. Supply the name of your internal interface in place of eth0 for your situation.

Q: I want to share my administrative workstation across multiple clusters. Is there anything special I need to do?

A: If each of your clusters is connected to a dedicated NIC on your administrative machine, you need to do nothing special. Configure a template for each cluster with approporiate values and all should work. If your clusters are connected through a dedicated 'virtual' interface (interface alias, under Linux for instance), there is a complication. packets broadcast to 255.255.255.255 are picked up by real interfaces only, not by alias interfaces. To force bootpd to report the correct server address in this case, you need to add an 'sa' entry to cluster templates in your bootptab file. See the provided sample bootp file for an example.

Light reading:

/etc/bootptab: A sample /etc/bootptab for use with Beowulf systems
IETF RFC 1534: Interoperation between DHCP and BOOTP